Gemma/[Gemma_3]Using_with_Ollama_Python_Inference_with_Images.ipynb (405 lines of code) (raw):
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "j4qMNV_533ls"
},
"source": [
"##### Copyright 2025 Google LLC."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "urf-mQKk348O"
},
"outputs": [],
"source": [
"# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iS5fT77w4ZNj"
},
"source": [
"# Gemma - Run with Ollama Python library\n",
"\n",
"Author: Onuralp Sezer\n",
"\n",
"* GitHub: [github.com/onuralpszr](https://github.com/onuralpszr/)\n",
"* X: [@onuralpszr](https://x.com/onuralpszr)\n",
"\n",
"Description: This notebook demonstrates how you can run inference on a Gemma3 model using [Ollama Python library](https://github.com/ollama/ollama-python). The Ollama Python library provides the easiest way to integrate Python 3.8+ projects with Ollama.\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_3]Using_with_Ollama_Python_Inference_with_Images.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FF6vOV_74aqj"
},
"source": [
"## Setup\n",
"\n",
"### Select the Colab runtime\n",
"To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma model. In this case, you can use a T4 GPU:\n",
"\n",
"1. In the upper-right of the Colab window, select **โพ (Additional connection options)**.\n",
"2. Select **Change runtime type**.\n",
"3. Under **Hardware accelerator**, select **T4 GPU**."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4tlnekw44gaq"
},
"source": [
"## Installation"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AL7futP_4laS"
},
"source": [
"Install Ollama through the offical installation script."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MuV7cWtcAoSV"
},
"outputs": [],
"source": [
"!sudo apt-get install pciutils\n",
"!curl -fsSL https://ollama.com/install.sh | sh"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FpV183Rv6-1P"
},
"source": [
"Install Ollama Python library through the official Python client for Ollama."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "0Mrj29SH-3OD"
},
"outputs": [],
"source": [
"!pip install -q ollama"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iNxJFvGIe48_"
},
"source": [
"## Start Ollama\n",
"\n",
"Start Ollama in background using nohup."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "5CX39xKVe9UN"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"nohup: redirecting stderr to stdout\n"
]
}
],
"source": [
"!nohup ollama serve > ollama.log &"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6YfDqlyo46Rp"
},
"source": [
"## Prerequisites\n",
"\n",
"* Ollama should be installed and running. (This was already completed in previous steps.)\n",
"* Pull the gemma3 model to use with the library: `ollama pull gemma3:4b`\n",
" * See [Ollama.com](https://ollama.com/) for more information on the models available."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "oPU5dA1-B5Fn"
},
"outputs": [],
"source": [
"import ollama"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"id": "NE1AWlucBza_"
},
"outputs": [
{
"data": {
"text/plain": [
"ProgressResponse(status='success', completed=None, total=None, digest=None)"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ollama.pull('gemma3:4b')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KL5Kc6HaKjmF"
},
"source": [
"## Inference with Text ๐ and Image ๐ผ๏ธ\n",
"\n",
"Run inference using Ollama Python library."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-CjTl1ajarEE"
},
"outputs": [],
"source": [
"!curl -A \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36\" -o pexels-adria-masi-461420600-27372369.jpg \"https://images.pexels.com/photos/27372369/pexels-photo-27372369.jpeg?cs=srgb&dl=pexels-adria-masi-461420600-27372369.jpg&fm=jpg&w=1280&h=856\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MclgY4tLbpch"
},
"source": [
"### Image Preview (Optional) ๐ผ๏ธ"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2mQbRM3JbZt2"
},
"outputs": [],
"source": [
"import cv2\n",
"from google.colab.patches import cv2_imshow\n",
"cv2_imshow(cv2.imread('pexels-adria-masi-461420600-27372369.jpg'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HN0nrhpmFUUB"
},
"source": [
"### Generate"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VvBB_XoObyx1"
},
"outputs": [],
"source": [
"import ollama\n",
"\n",
"res = ollama.chat(\n",
"\tmodel=\"gemma3:4b\",\n",
"\tmessages=[\n",
"\t\t{\n",
"\t\t\t'role': 'user',\n",
"\t\t\t'content': 'Describe this image:',\n",
"\t\t\t'images': ['./pexels-adria-masi-461420600-27372369.jpg']\n",
"\t\t}\n",
"\t]\n",
")\n",
"\n",
"print(res['message']['content'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Vh_9LD_ocl6t"
},
"outputs": [],
"source": [
"!wget https://ollama.com/public/blog/wordart.jpg"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "utyVRlIYFvdm"
},
"source": [
"#### Async client with OCR Usage ๐\n",
"\n",
"To make asynchronous requests, use the `AsyncClient` class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JVvKQE0XF2kh"
},
"outputs": [],
"source": [
"import asyncio\n",
"import nest_asyncio\n",
"from ollama import AsyncClient\n",
"\n",
"nest_asyncio.apply()\n",
"\n",
"\n",
"async def generate():\n",
" \"\"\"\n",
" Asynchronously generates a response to a given prompt using the AsyncClient.\n",
"\n",
" This function creates an instance of AsyncClient and sends a request to generate\n",
" a response for the specified prompt. The response is then printed.\n",
" \"\"\"\n",
" # Create an instance of the AsyncClient\n",
" client = AsyncClient()\n",
"\n",
" # Send a request to generate a response to the prompt\n",
" message={\n",
" 'role': 'user',\n",
" 'content': 'What does the text say ?',\n",
" 'images': ['./wordart.jpg']}\n",
"\n",
"\n",
" async for part in await AsyncClient().chat(model='gemma3:4b', messages=[message], stream=True):\n",
" print(part['message']['content'], end='', flush=True)\n",
"\n",
"# Run the generate function\n",
"asyncio.run(generate())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9V2TY2KQesS1"
},
"outputs": [],
"source": [
"!curl -A \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36\" -o pexels-pixabay-60582.jpg \"https://images.pexels.com/photos/60582/newton-s-cradle-balls-sphere-action-60582.jpeg?cs=srgb&dl=pexels-pixabay-60582.jpg&fm=jpg&w=1920&h=1346\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WFyi_zzWAwe7"
},
"source": [
"### Object Counting ๐งพ"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UN20MoSv_S76"
},
"outputs": [],
"source": [
"from ollama import chat\n",
"\n",
"# Start a conversation with the model\n",
"messages = [\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": \"How many Balls on the cradle?\",\n",
" \"images\": [\"./pexels-pixabay-60582.jpg\"]\n",
" },\n",
"]\n",
"\n",
"# Get the model's response to the message\n",
"response = chat(\"gemma3:4b\", messages=messages)\n",
"print(response[\"message\"][\"content\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cDr8VzvGIXFC"
},
"source": [
"## Conclusion ๐\n",
"\n",
"Congratulations! You have successfully run inference on a Gemma3 model using the Ollama Python library with VLM capabilities. You can now integrate this into your Python projects."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "[Gemma_3]Using_with_Ollama_Python_Inference_with_Images.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}